1. Field of the Invention
The present invention relates to image processing apparatuses and methods, recording media, and programs. More specifically, the present invention relates to an image processing apparatus and method, a recording medium, and a program for accurately calculating motion vectors on a pixel-by-pixel basis and for switching the method of generating pixels in accordance with the amounts of error that occurs in pixels generated based on the motion vectors calculated.
2. Description of the Related Art
Techniques for generating new pixels based on pixels of an existing image to convert the resolution or field frequency of the image are becoming common.
In one known method for generating pixels of a moving picture, motion vectors are used. A motion vector indicates a direction and a distance of movement of a pixel having temporally the same pixel value in a moving picture or the like.
FIG. 1 shows the construction of an image processing apparatus 1 according to the related art, which converts the number of scanning lines using motion vectors.
The image processing apparatus 1 includes image memories 11-1 and 11-2, a block-motion-vector detector 12, and an image synthesizer 13. The image processing apparatus 1 receives input of interlaced image signals that are obtained via a video source, supplied from, for example, a television (TV) tuner or a digital versatile disc (DVD) player (not shown), and that are digitized by an analog to digital (AD) converter, converts the number of scanning lines of the input data, and outputs the resulting data.
Each of the image memories 11-1 and 11-2 stores input field data corresponding to one field, and outputs the field data with a delay corresponding to one field. Thus, the image memory 11-1 supplies delayed field data that is delayed by one field relative to a present field to the image synthesizer 13. The image memory 11-2 supplies previous-delayed-field data that is further delayed by one, field. Thus, the image memory 11-2 supplies field data that is delayed by two fields in total (i.e., previous-delayed-field data) to the block-motion-vector detector 12 and the image synthesizer 13.
The block-motion-vector detector 12 obtains present-field data and previous-delayed-field data supplied from the image memory 11-2, detecting motion vectors based on correlation between these pieces of field data of two fields. More specifically, for example, the block-motion-vector detector 12 detects motion vectors on a block-by-block basis by block matching. In block matching, for example, a reference block consisting of a predetermined number of pixels in a present field is set, and a search block in a previous delayed field having the same size as the reference block is set. Then, pixels in the search block are sequentially extracted while moving the search block sequentially in the previous delayed field, and the sum of the absolute values of differences between pixels at corresponding positions in the reference block and the search block is calculated with the search block at each position. Then, a motion vector is calculated based on positional relationship of the search block and reference block that minimizes the sum of the absolute values of differences.
The image synthesizer 13 obtains present-field data, delayed-field data and previous-delayed-field data supplied from the image memories 11-1 and 11-2, respectively, and block motion vectors supplied from the block-motion-vector detector 12, synthesizes an image based on these pieces of information, converts the number of scanning lines, and outputs the resulting data to a subsequent stage.
Next, the operation of the image processing apparatus shown in FIG. 1 will be described.
First, the image memory 11-1 stores the first field data. At a next timing, the image memory 11-1 supplies the field data stored therein to the image memory 11-2 and the image synthesizer 13 as delayed-field data. At a further next timing, the image memory 11-1 supplies delayed-field data stored therein to the image memory 11-2 and the image synthesizer 13. Furthermore, the image memory 11-2 supplies the previous-delayed-field data stored therein to the block-motion-vector detector 12 and the image synthesizer 13.
At this time, the block-motion-vector detector 12 calculates motion vectors on a block-by-block basis by block matching using the present-field data and the previous-delayed-field data, and supplies the motion vectors to the image synthesizer 13.
The image synthesizer 13 generates pixels by inter-field interpolation using pixels on the present field data and the previous-delayed-field data, the pixels corresponding to points where motion vectors passing through pixels on scanning lines that must be newly generated for a progressive image of the delayed field supplied thereto cross the present field and the previous delayed field, thereby converting an interlaced delayed-field image into a progressive image and outputting the progressive image.
Furthermore, for example, according to techniques disclosed in Japanese Unexamined Patent Application Publication No. 5-219529, motion vectors are detected by detecting movement of color-difference signals as well as movement of luminance signals so that motion vectors can be detected accurately.
However, according to the method of detecting motion vectors by block matching, the accuracy of detecting motion vectors decreases when the block size is large, but the accuracy of detecting motion vectors does not improve even when the block size is chosen to be smaller than a certain value. Furthermore, in block matching, the amount of computation becomes huge when the block size is small, resulting in excessive computational cost.
Furthermore, in order to meet incompatible requirements of improving the accuracy of detecting motion vectors and reducing computational cost to achieve real-time processing, motion vectors are calculated on a basis of individual blocks having a predetermined size. Thus, when pixels having two or more motion vectors exist in a search block in which a motion vector is to be detected, in an image consisting of pixels generated by interpolation, distortion could occur on a block-by-block basis, mismatch could occur between pixels generated by interpolation and peripheral pixels, or the spatial resolution could be apparently degraded since the same motion vectors are used on a block-by-block basis.